Relative Radiometric Normalisation of Unmanned Aerial Vehicle Photogrammetry‐based RGB Orthomosaics

The problem of brightness differences between images the same scene is important to field unmanned aerial vehicle (UAV) photogrammetry and affects both aesthetics interpretation final product. This can be caused by changes in positions camera or sun, as well weather conditions. article deals with varied image latter. Relative radiometric normalisation (RRN) acquired RGB imagery used diminish effects this phenomenon improve visual quality resultant presented algorithm considers specificity UAV-acquired data. It utilises their relationships group similar images, choose references perform RRN via histogram matching. method robust fully automatic. Validation performed on two independent datasets confirms its effectiveness qualitatively (improving appearance orthomosaic) quantitatively. Le problème des différences de luminosité entre les d'une même scène est dans le domaine la photogrammétrie par drone, affectant à fois l'esthétique et l'interprétation du produit final. Ce peut être dû changements position l'appareil photo ou soleil, ainsi que conditions météorologiques. Cet traite variation causée ces dernières. La radiométrique relative l'imagerie RVB utilisée pour diminuer l'impact ce phénomène améliorer qualité visuelle obtenu. L'algorithme présenté tient compte spécificité données acquises drone. Il utilise leurs relations regrouper similaires, choisir références effectuer mise en correspondance histogrammes. méthode finale robuste entièrement automatique. validation effectuée sur deux jeux indépendants confirme son efficacité tant plan qualitatif (amélioration l'apparence l'orthomosaïque) quantitatif. Das Problem der Helligkeitsunterschiede zwischen Bildern derselben Szene ist auf dem Gebiet unbemannter Luftfahrzeuge Fotogrammetrie wichtig und beeinflusst sowohl die Ästhetik als auch Interpretation Endprodukts. Dieses kann durch Änderungen Kameraposition oder Sonneneinstrahlung sowie Wetterbedingungen verursacht werden. Dieser Artikel befasst sich mit unterschiedlichen Bildhelligkeit, das letztere wird. Die radiometrische Normalisierung von aufgenommenen RGB-Bildern wird verwendet, um Auswirkungen dieses Phänomens zu verringern Qualität resultierenden Produkts verbessern. Der vorgestellte Algorithmus berücksichtigt Besonderheit UAV-erfassten Daten. Es nutzt Bildpositionen ihre Beziehungen, ähnliche Bilder gruppieren, Referenzen auszuwählen über Histogramm-mabgleich durchzuführen. endgültige Methode vollautomatisch. Validierung an zwei unabhängigen Datensätzen bestätigt Wirksamkeit qualitativer (Verbesserung Erscheinungsbildes Orthofotomosaiks) quantitativer Hinsicht. Las diferencias brillo imágenes misma escena es un problema importante el campo fotogrametría vehículos aéreos no tripulados y afecta tanto a estética como interpretación del producto Este puede ser debido cambios las posiciones cámara o sol, así condiciones climáticas. artículo trata sobre variación debidos estos cambios. normalización radiométrica relativa adquiridas se utiliza para reducir los efectos este fenómeno mejorar calidad resultante. El algoritmo presentado considera especificidad datos adquiridos por UAV. Utiliza sus relaciones agrupar similares, elegir referencias realizar través comparación histogramas. método robusto totalmente automático. validación realizada dos conjuntos independientes confirma su eficacia cualitativa (mejorando apariencia mosaico ortoimagenes) cuantitativamente. 同一场景的影像之间的亮度差异问题对无人驾驶飞行器(UAV)摄影测量领域很重要，并影响到最终产品的美观和解释。这个问题可能是由相机或太阳的位置变化以及天气条件造成的。本文讨论的是由后者引起的影像亮度变化问题。对获取的RGB影像进行相对辐射归一化(RRN)用以减少这种现象的影响，并提高最终产品的视觉质量。所提出的算法考虑了无人机获取数据的特殊性。它利用影像的位置和它们的关系来分组类似的影像，选择参考，并通过直方图匹配来执行RRN。最终的方法是稳健的和全自动的。在两个独立的数据集上进行的验证证实了它在质量上(改善正射影像的外观)和数量上的有效性。 UAV changing normalization orthomosaic. utilizes them, references, Acquiring high-resolution partial cloud cover lead clearly visible unevenness resulting increases survey mission duration particularly acute when performing double grid flights. obvious solution conduct flights under either overcast cloudless sky. However, practice, it not always possible select date ensure optimal flight conditions, especially specific climatic (Wang al., 2019). In such cases, generated orthomosaic may differ significantly from that which desired. because have impact further spatial analyses. minimise phenomenon. There are approaches, absolute relative. approach requires ground-reflectance measurements sensor calibration. only costly, but also often impossible apply, particular for full-scale images. Consequently, preferred. uses image-based information normalises digital numbers (DNs) reference image. heterogeneities concerns different altitudes various phenomena, during data acquisition, bidirectional reflectance distribution function (BRDF) defects. common, current theme satellite research (Bai 2018; Santra 2019; Latte Lejeune, 2020; Yan 2021; Yin 2021). These studies most concern area at times even using sensors (Ghanbari Occasionally, these focus generation seamless orthoimage mosaics (Zhang 2017; Syariz Liu 2021), relatively large areas imaged one pass. temporal scales mean heterogeneity among cannot solved tools. photogrammetry, topic arises times, sensors, (Chandelier Martinoty, 2009; Pan 2010; Gehrke Beshah, 2016). Such occurs after considering calibration, atmospheric correction influence BRDF. proposed Beshah (2016), determined selected, evenly distributed tie points. Radiometric based adjusting standard deviation (histogram stretching) separately each channel. scale strip width typically larger case than photogrammetry. Also, over given take much longer plane. means necessary more UAVs (Wierzbicki 2015). addition, slightly context stability subsequent geometry lower, perspective rapidly general (non-metric) lower. For reasons, problems bigger completely those observed imagery. At time, some present is, correction, significant surveys. As result, feasible apply procedures directly Therefore, there need adapted Currently, addressed literature mainly relation multispectral has been focused calibration transformation remote sensing (Mafanya Cao Porto Shin Suomalainen Shadows obstacles ground clouds factors determining reflectances derivative indicators (Aboutalebi researchers sought develop algorithms detecting shaded (Adeline 2013). Another negative variable solar irradiance determination use multiple surveys BRDF precise corrected (Tu 2018). Honkavaara al. (2012) overlaps block linear empirical model correct mosaic surface reflectance. was subsequently developed paper (Honkavaara Khoramshahi, sensing, illumination influences Moreover, improving radiometry image-processing tools does affect triangulation results negatively (Dominik, 2014; Kedzierski Wierzbicki, Nevertheless, almost affected illumination. Thus, still order achieve goal applied processing seeks normalise radiometrically source due A covered Due acquisition mode create somewhat ones already described literature. simple use, automated, tuned specifically available together, employing if necessary. workflow includes three steps designed connections within existing establish homogenous Input modified properties output original pipeline (Fig. 1(a)), additional process introduced (DSM) before orthorectification 1(b)). first stage required establishment dataset. Here, components were positioning points them. Knowledge regarding then transferred two. lighting characteristics grouped together. biggest became all excluded aligned it. iterative equalisation, where match adjust main group. study orthomosaic, perceptual colour space – CIELab utilised stage. Although Global Navigation Satellite Systems (GNSS) inertial navigation systems provided about rotation accurate establishing bundle adjustment. adjustment DSM 2). contained adjusted (external orientation elements), sparse point cloud, exact corresponding DSM. Then, bound pairs DN overlapping parts provided. Tie geometrical could basic pairs. Unfortunately, every pair shared projection outline made base-pair set created sufficient overlap. overlap threshold algorithmic parameter depended algorithm, should lower 50%. Next, add pair. difference calculated band. band respective fewer assumed threshold. 50. considered processing. step 3), divided into groups: corrected. To enable division, L* base set. Continuous might disturbed changes. groups disconnected other. random connected paths. path errors, DNs produced paths would different. find single, value An procedure implemented while minimising error. One randomly chosen reference, pairs' treated observations least square solution. number equations corresponded variables equal system built calculate corrections initial guesses brightness. coordinates: centre XY 4). plane fitted “coordinates” sample consensus (RANSAC) (Fischler Bolles, 1981). inlier distance along levels smaller grouping All inliers classified remaining (images) part equalisation 5). iteration, first, set, consisting opposite chosen. Since include done prevent redundant corrections. choice maximum Histogram matching retained pair; matched 6). Both histograms look-up tables entire When complete, reclassified started again beginning repeated long conducted areas. area, denoted JAW, located Jaworzno (50°10′ N, 19°20′ E). second marked acronym KRC, close Korczowa (49°57′ 22°59′ photogrammetric Two series carried out JAW area. occurred July 2019 (JAW072019) February 2020 (JAW022020). KRC mapped once December (KRC122019). network control (GCPs) established measured GNSS real-time kinematic method. fixed-wing BIRDIE (FlyTech UAV, Krakow, Poland) JAW022020 KRC122019 series. platform Sony DSC-RX1RM2 equipped Carl Zeiss Sonnar T* 35 mm f/2 lens. JAW072019 series, DJI S1000 octocopter (DJI, Shenzhen, China) ILCE A7R FE f/2.8 ZA lens (Sony, Tokyo, Japan) Zenmuse Z15-A7 gimbal (DJI) acquire autonomously prior planning. sizes length, missions several parts. measurement equipment specifications, relief, land sampling (GSD) target size, planned forward side overlaps, previously diagnosed on-board instrument errors (Ćwiąkała, 2019) datasets, consisted parallel strips speed time needed save photo. dataset, photos taken arranged transversely longitudinally boundaries desired low altitude double-grid allowed coverage maintained. parameters associated summarised Table I. 19 9:44–10:15 9:47–10:16 10:31–11:04 11:20–11:41 29 12:39–12:56 13:05–13:22 13:33–13:51 14:10–14:27 14:35–14:45 collected Agisoft Metashape Professional v. 1.7.2 (Agisoft, St Petersburg, Russia). Aerotriangulation resolution (images size). alignment quality, outliers removed gradual selection tool software. included re-alignment parameters. evaluated positively 7(c)). orthomosaics. Variation seen KRC122019. influenced soil moisture 7(a)). shadows quite vivid 7(b)). Changes coincided direction striking chequerboard pattern seen. intensity throughout stable survey, burdened above. Data displacement verify effect accuracy. comparison inside contour red line Fig. 7. written Python custom package. Basic II orthomosaics shown 8. processed, uniform qualitative quantitative verification validate proposal. Qualitative perusal note changes, improvements deteriorations. Quantitative similarity checks validation. During stage, accuracy vector automatically assessed. dataset exhibits mostly coincide patterns (Figs. 9(a) (b)). dark light patches fields, intuitively expected. valid, they moisture, retain lighter disappear 8(a) 9(c)). Processing successfully removes 8(b)). distinct north-west, central eastern parts, excessively bright southern part. borders clear visible. contribute effect. Optical factors, vignetting function, visible, results. plays role recorded sunlight intensities follow tilts sun. Differences neighbouring ignored, small become noticeable row. On other hand, full fit horizontal negligible differences. too whether any improvement deterioration occurred, Bhattacharyya (Bhattacharyya, 1943). 0–1 range, 1 lowest 0 highest. score none changed excluded. Then distances, paired t-test determine what significant. 1627 Similarity 10. seem (L* b*) deteriorations (a*). While t-tests (Table III) demonstrate statistically significant, according size Cohen's formula (Sawilowsky, 2009) change (L*), medium (a*) very (b*). 1968 compared. III show bands. suggests importance b* bands a* way check processing, checks. this, squares (0.2 m) placed reprojected them calculated. per-square deviations received values compared significance. Grid 190 209 JAW022020. 11) shows majority reduced. Formal testing bands, IV). Calculated Dataset JAW022019 far greater distributions visibly converge closer zero decreases. influence, last implementation raster-based displacements. purpose, density (Puniach 2021) UAV-derived normalisation. underground mining hard coal deposits. analysed 1(c). analyses (base series) (aligned series). concerned pixel weights. case, weights segmenting classes: vegetation assigned weight non-vegetation thresholding. modification outlier removal process. filter interquartile range vectors consistent identified vicinity. root error (RMSE) assessed approximately 200 manually coordinate estimated (10 mm). displacements 1.5 pixels (15 Based (before normalisation), terrain automatically. densities accuracies designated V. general, automatic 1.5% raw removal) 5% removal. percentage correctly increases. ambiguous. RMSE post-normalisation (2D) 16 21 VI pre- noted roughly level values. JAW022020post-normalisation component post-radiometric 12) distributions. usually do exceed limit displacements, 30 (1.96 * 15 mm) 95% confidence level. analysis 13). No relationship noticed (darkening/brightening fragment contains methodology (Gehrke method, assumptions Connection bearing rapidly, proximity considered. utilising connections. prioritises largest groups, turn ensures minimisation RRN. improved. describe work case. influential gets complicated. Vignetting phenomena cause generate nonlinear minimised illumination, complicates separate study. mitigate problem. That local planes, polynomial spline surfaces. will works. stretching commonly production gives good simplified assumes (Glover, 2011). imagery, assumption true. Furthermore, though invasive chances overly degrading mitigated. tested subset latter gave better results, decision manipulation differs solutions. Instead points, signal occurrence computationally costly solution, provides object scene, unlike approach. occur naturally ground, unlikely artificial tall objects. limited variability overcome dense calculations. prolong calculations radically require algorithms. By correlation channels, facilitates identification problematic emphasised band, combination R, G B just colour. encouraging heavy intrusion overcorrection. presents highly From view product, best homogeneous, uniformly overcast, performance exposure unavoidable, enables relationships, prevalent exposures assessments confirm reducing heterogeneity. indicate upon project partly supported programme “Excellence Initiative Research University” AGH University Science Technology SqM Farm 34009-17-1303 funded Danish Green Development Demonstration Programme. Open access funding enabled organized ProjektDEAL.